Means for using microstructure of materials surface as a unique identifier

ABSTRACT

A method and apparatus for the visual identification of materials for tracking an object comprises parameter setting, acquisition and identification phases. The parameter setting phase comprises the steps of defining acquisition parameters for the objects. The acquisition phase comprises the steps of digitally acquiring two-dimensional template image of an object, applying a flattening function and generating downsampled template version of the flattened template and storing it in a reference database with the flattened template. The identification phase comprises the steps of digitally acquiring a snapshot image, applying the flattening function and generating one downsampled version, cross-correlating the downsampled version of the flattened snapshot with the corresponding downsampled templates of the reference database, and selecting templates according to the value of the signal to noise ratio, for the selected templates, cross-correlating the flattened snapshot image with the reference flattened template, and identifying the object by finding the best corresponding template.

The present application concerns the visual identification of materialsor documents for tracking, tampering, and anti-counterfeiting purpose.

STATE OF THE ART

This patent addresses the problem of counterfeiting, tampering andtraceability. Each of these three security issues are generallyaddressed using a specific approach:

-   -   For counterfeiting, approaches are based on special markings        (like holograms, DNA codes, optically variable inks,        Cryptoglyph, etc) where the operator checks if the marking is        there or not. Such solutions are useful mainly against        counterfeiting (and tampering in some cases) and their security        solely relies on the complexity to counterfeit the mark.    -   For tampering, solutions are either based on a redundant        encoding strategy or on a tamper evident approach. Redundant        security is based on a double encoding of critical information        (like the name on a passport which is also encoded in the        attached magnetic strip or the hash of a text which is printed        on a document which should not be modified). Tamper evidence can        also be achieved using various physical or chemical means which        enable to detect modifications performed to a document or to a        package. Such means include special paper treatments which        enable it to be immediately colored by solvents or ultrasonic        detection systems which are capable to detect an overlay of a        thin piece of paper.    -   Traceability is achieved using a unique identification of each        item. This type of system typically uses a central database        which maintains a record of information for each numbered item.        It should be noted that unique identification potentially        enables to address all security issues like counterfeiting,        tampering and diversion. The following section details the state        of the art of this approach

For carton and paper application, unique identification is oftenperformed by ink-jet printing (DOD or continuous) of an alphanumericstring or a barcode.

In the case of tampering detection, the goal is to guarantee that dataprinted in clear text on a document has not been illegally modified (forinstance a check value is increased). One simple way to reach thisobjective is to uniquely identify the document (using barcode orhexadecimal string) with an identification number printed on thedocument. This number gives access through a look-up table to the dataalso printed in clear text on the document. It is then possible to checkif the data printed on the document matches with the data stored in theloop-up table (Optical Character Recognition may be used for automatingthe comparison process). There exists many way to implement this ideafor specific problems, for instance a solution for integrity check ofidentity documents is described in U.S. Pat. No. 6,920,437, another forpassport documents is given in U.S. Pat. No. 6,565,000.

Traceability is particularly important for finding parallel import ofgoods, but also for other security purposes, in the following we chooseto focus on the specific diversion problem for the sake of clarity.Unique identifiers for each package sent to a given country A and by agiven distributor, are listed in a database. In case of gray market, thegood is re-imported in a country B. By using this code it is thenpossible to trace back the origin of the good. A particular systemarchitecture (comprising a central server, database and clientapplications) of such an approach using central database is described inU.S. Pat. No. 6,922,687, another system architecture is given in U.S.Pat. No. 6,547,137 (where the identifier is printed on a label attachedto the product). For the pharmaceutical industry, this solution can alsobe implemented by marking the identifier directly on a label or a pillas described in U.S. Pat. No. 6,776,341. For the electronic industryU.S. Pat. No. 6,629,061 describes a solution where the identifier isprinted on the circuit board and embedded in the memory device (anotherapproach for fighting gray market in this industry is given in U.S. Pat.No. 6,459,175 with power supplies). For the clothing industry, U.S. Pat.No. 6,461,987 describes a solution where unique identification isobtained by the means of micro label strips.

In the case of counterfeit detection, identifiers of all the producedpackages or documents are kept in a central database. For each productit is then possible to interrogate the database and know:

-   -   If the identifier belongs to the database. If so, then it is the        proof that this is a valid identifier, i.e. it has not been        invented by a counterfeiter (identification numbers are randomly        chosen using a secret algorithm).    -   If another request has already been sent for the same        identifier. If so, it would prove that several counterfeit        copies of the same product are circulating.

Some of the patent applications listed in the above section ontraceability also describe solutions for counterfeit detection. Hence,counterfeit is also described in U.S. Pat. No. 6,776,341 (using labels,as described above) and U.S. Pat. No. 6,461,987 (using micro labelstrips for the clothing industry, see above).

The marking of the code can be performed either by means of printing orengraving as described in U.S. Pat. No. 6,706,314 (in this case a lasercan be used for reading) for traceability and anti-counterfeitapplications. It can also be used special light sources with materialreacting to specific wavelengths. Although this approach is generallymore used as a yes/no answer (since it is capable of generating a verylimited number of different identifiers), patents U.S. Pat. No.6,384,409 and U.S. Pat. No. 6,030,657 (where the material is abiological marker) mention fighting gray market using this approach. Thesame stands for the analog magnetic reader described in U.S. Pat. No.5,975,581.

The unique identification method described above is therefore a powerfulapproach that enables to solve three different security issues:tampering, tracking and counterfeiting.

So far, we have investigated only unique identification relying on amarking of the product. There exists also a totally different approachenabling unique identification without marking (both approaches aredescribed in FIG. 1).

Indeed, it is also possible to measure precisely some features of adocument or product and use it to characterize the product uniquely. Forinstance, Ingenia Technology discloses an invention where the microtopology of carton and paper is measured using a coherent light(typically produced by a laser) and used for unique identificationpurposes in GB0418138.4, GB0418173.1, GB0418178.0 and GB0509635.9. Thistechnology may be directly embedded in printer devices (as described inPCT/GB2005/000903). This technology can basically be used on any chaoticsurface, using an array of laser sources enabling to probe the materialsurface at various incidences (as described in PCT/GB2005/000922). Asimilar approach was also described in application GB2221870 from De LaRue Company Plc, where the scattering of a coherent light was used fordetection. Another solution is described in U.S. Pat. No. 6,584,214 bythe MIT where the whole 3D chaotic structure of a material is used togenerate a unique identifier. The 3D structure is acquired using deviceswhich are based on coherent light (for transparent material) orultrasound and X-rays (for non-transparent materials). Another approachusing ultrasonic measurement is described in U.S. Pat. No. 5,454,045,where features (either macroscopic or microscopic) are measured inside amaterial, stored and subsequently compared with new measurements formatch test. However, all of these approaches use specific acquisitiondevices in order to ease the detection of unique features required forunique identification applications. The current application describessolutions for using standard imaging devices.

In published patent US20050075984 (also in US20030014647-A1) an originalmethod based on random set of micro bubbles inserted in a transparentmedium is described. The detection is based on measurement of shadowsand reflections to determine a unique signature for each set of bubbles.The transparent medium is then physically affixed to the product ordocument to be identified. This approach is unusual as it is somehowbetween the two processes described in FIG. 1: on the one hand it is ananalog random process but on the other hand it requires to be physicallyapplied on the product which is conceptually the same approach asprinting out a serial number. Another family of solutions is based onthe creation of a digital signature using the random and chaotic natureof materials. Such a digital signature can be used for authenticationpurposes, for encryption purposes or for tampering detection.Applications related to authentication are for instance disclosed inPCT/BY99/00011 (where the signature is encrypted and printed on thematerial itself). Two patent applications disclosed by the companySignoptic Technologies focus on the generation and use of a digitalsignature using material microstructure. In document WO2005/122100,different applications are described where the signature is used toencrypt data. The document WO2006/078651 focuses specifically onsignatures obtained from fibrous materials for authentication purposes.

However, those approaches do not address the so-calledmass-serialization application. This strategy is used today in manyindustries and consists in individually marking each product whichenables to track of each of them. Such marks typically representvariable barcodes or alphanumeric codes which are digitally printed(using Drop On Demand on continuous inkjet) or engraved (using forinstance laser marking). The solutions disclosed in the currentapplication enable to provide all the functionality offered bymass-serialization but without the need of any marking, by only usingimages of their microstructure. It describes in particular solutions inorder to provide automatic high speed recording and identification ofproducts.

SHORT DESCRIPTION OF THE INVENTION

The invention describes how to perform a reliable and fastidentification or traceability of a product by uniquely using an opticalimage of the microstructure of its surface. The randomness of thismicrostructure, from one product to the other is used to identifyuniquely each product. The identification is performed by a matchbetween the acquired image and a set of previously recorded images ofthe microstructure of all the products of a given production line.

In the following, some cases of materials are specifically analyzed butthe described method is not limited to these materials and can beapplied easily to other types of materials and detected features for anyperson skilled in the art of image processing and physics of materials.

The present invention proposes a method to automatically identify anobject comprising a parameter settings phase, an acquisition phase andan identification phase, the parameter setting phase comprising thesteps of:

-   -   defining for a given set of objects, a resolution, a type of        non-coherent light illumination and a location, called region of        interest, for the acquired image for which the object's        microstructure image contains noise,        the acquisition phase comprising the following steps, for each        object to be later identified:    -   digitally acquiring a two-dimensional image of the object        according to parameter settings through sampling on a uniformly        spaced orthogonal grid of at least one color component,    -   applying a flattening function on said template in order to        produce a flattened template by removing macroscopic color        variations,    -   generating at least one downsampled template version of the        flattened template,    -   storing in a reference database the downsampled template version        and the flattened template,        the identification phase comprising the following steps, for the        object to be identified:    -   digitally acquiring a two-dimensional snapshot image according        to the same parameters as the template image,    -   applying to the snapshot image the same flattening function as        the one applied to the template image in order to produce a        flattened snapshot image,    -   generating at least one downsampled version of the flattened        snapshot image,    -   cross-correlating the downsampled version of the flattened        snapshot image with the corresponding downsampled templates        version stored in the reference database, and selecting the        templates according to the value of the signal to noise ratio of        said cross-correlation,    -   for the selected templates, cross-correlating the flattened        snapshot image with the flattened template stored in the        reference database, and thus identifying the object by finding        the best corresponding template which signal to noise ratio        value of said cross-correlation is above a predefined threshold.

The main fields of applications of the presented technology includetracking, identification and counterfeit detection. Three versions ofthe technology are described, each covering parts or all of the fieldsof application:

-   -   The first level is named “unique fingerprint” in this document.        With this technique it is possible to identify, to track        uniquely and to detect counterfeiting for some valuable item or        document. This technology is based on the unique microstructure        of the surface of each item.    -   The second level is named “microscopic fingerprint” in the        present document. With this technology it is possible to detect        counterfeited items but not to trace them or to identify them        uniquely. This technology is based on the microstructure of the        object used to make the item, for example a mould or an offset        sheet, depending on the material in which the item is made.    -   The third level is named “macroscopic fingerprint” in the        present document. This allows detecting coarsely counterfeited        items, such as fake medicines that are sold on the internet.        This technology is based on the macroscopic structure of the        item to protect.

Examples of usage of this technology are watch traceability andanti-counterfeiting, authentication and anti-counterfeiting of paperdocument, label or package, authentication of containers, authenticationof automotive parts, authentication of electric pieces, such as circuitbreaker, retrieval of advertisements in newspaper pages and so on. Itcan be used for example in watch industry, pharmaceutical industry, carindustry, electronic industry, food industry or newspaper industry andmore generally for any industry which requires identification ortraceability of items without marking. This technology has been named“fingerprint” as it is based on concepts which are similar to those usedfor the human fingerprint technologies. In fact, the inside surface ofhands and feet of humans (all primates in fact) contains tiny ridgeswith furrows in between which are unique for each individual. This isalso true for the majority of materials like wood, paper, metal, resin,plastics or other coated materials. The concept of this application isactually not limited to images of materials. It can be also applied toany noisy signal which constitutes a signature, including but notlimited to magnetic signal (magnetic spatial variations around a magnetfor instance), electric signal (electric field generated by anelectronic component), electromagnetic signal (spectrum of light emittedby an object), etc

SHORT DESCRIPTION OF THE FIGURES

The invention will be better understood thanks to the attached drawingsin which:

FIG. 1: Two different processes enabling a unique identification.Process 1: an analog random process 11 (like the microstructure of apaper) is digitized with 12 and the resulting data is stored an uniqueidentifier 13. In process 2, a generator 14 provide a digital random (orpseudo-random) value that is stored as a unique identifier 15 andfinally printed 16.

FIG. 2: Picture a magnified portion of office paper showing theimportant randomness of the paper structure. The height of the pictureis approximately 1 mm. The image has been processed to enhance thecontrast.

FIG. 3: A portion of a metallic logo of a back of watch. The height ofthe image is approximately 1 mm. The picture has been processed toenhance the contrast. Picture was taken with an office digital scannerat 4,800 dpi.

FIG. 4: A portion of molded plastic. The height of the picture isapproximately 5 mm. The image has been processed to enhance contrast.Picture was taken with as scanner at 2,400 dpi.

FIG. 5: A part of a printed area on a paper label. The height of thepicture is approximately 5 mm. The image has been processed to enhancethe contrast. Picture was taken with a scanner at 2,400 dpi.

FIG. 6: A part of a coated dial of a watch. The height of the picture isabout 1 mm. The image was processed to enhance contrast. Picture wastaken with a microscope at about 10,000 dpi.

FIG. 7: A part of a resin item. The size of the picture is 0.35 mm×0.35mm. Picture was taken with a microscope at 20,000 dpi. The image wasprocessed to enhance contrast.

FIG. 8: A part of a laser engraved metal. The size of the picture is0.11 mm×0.11 m. Picture was taken with a microscope at 30,000 dpi. Theimage was processed to enhance contrast.

FIG. 9: A part of laser engraved metal. 91 shows the beginning of theengraving and 92 shows the pulsed part.

FIG. 10: A part of homogenous metal. The size of the picture is 2.2mm×2.2 mm. Picture was taken with a digital camera at 1445 dpi. Theimage was processed to enhance contrast.

FIG. 11: A part of non molded plastic. The size of the picture is 2.2mm×2.2 mm. Picture was taken with a digital camera at 1445 dpi. Theimage was processed to enhance contrast.

FIG. 12: Diagram showing how the captured data can be increase toprevent possible false positive or false negative detections.

FIG. 13: Contours of the same text printed with industrial rotogravuretechnology (scale ˜1 mm height) for two successive samples of the sameproduction batch.

FIG. 14: Image of a uniform tint area of color printed in offset (scale˜1 mm for image height).

FIG. 15: Difference between structure of the chemically engraved logo oftwo different watches from the same model. Picture has been processed toenhance contrast. The field of view is of about 5 mm and the dpi is4,800. 151 is the engraved logo of one watch. 152 is the engraved logoof the other watch.

FIG. 16: Picture showing how to retrieve the angle of rotation of anitem. 161 is the Fourier transform of a specific item. 162 is theadditions of the values on an axis of 161 at each angle.

FIG. 17: Schema of image acquisition with diffuse light and 2D CCD. 171is the image acquisition device. 172 represents the diffuse light, 173is the light box and 174 is the item to verify.

FIG. 18: Schema of image acquisition with specular light. 181 is theimage acquisition device, 182 represents the specular light, 183represents the reflected light and 184 is the item to verify.

FIG. 19: Schema of image acquisition with diffuse light and 1D CCD. 191is the item to verify, 192 is a sheet of glass, 193 is a moving light,194 is a moving mirror, 195 is a fixed mirror and 196 is a 1D CCD.

FIG. 20: Extension of the imaging device of FIG. 51. 201 is the maincomponant as described in FIG. 51. 202 is a excrescence speciallydesigned to put any item on it. 203 is the item to verify.

FIG. 21: Picture of a coated dial observed with specular axial light.Picture has been processed to enhance the contrast. The field of view isabout 2×3 mm and the dpi is about 10,000.

FIG. 22: Picture of the same dial than in FIG. 21 but taken with a 1DCCD diffuse light. Picture has been processed to enhance the contrast.The field of view is about 5 mm and the dpi is 4,800.

FIG. 23: Picture of a coated dial observed with specular axial light.Picture has been processed to enhance the contrast. The field of view isabout 2×3 mm and the dpi is about 10,000.

FIG. 24: Picture of the same dial than in FIG. 23 but taken with a 1DCCD diffuse light. Picture has been processed to enhance the contrast.The field of view is about 5 mm and the dpi is 4,800.

FIG. 25: Schema depicting the difference between the three levels offingerprint technology. 251 is the axis for the technology, 252 is theaxis for the dpi. 253 represents the macroscopic fingerprint, 254represents the microscopic fingerprint and 255 represents the uniquefingerprint. White area are best area for the given technology whereasgray area are possible extensions of the area for the given technology.Resolutions in dpi corresponding to each area depends on the materialsproperties.

FIG. 26: Picture of two different back of watches that are stamped. Thefield of view is about 0.4×0.6 mm for a dpi of 10,837. 261 is thestamped structure of one watch, 262 is the one from the other.

FIG. 27: Picture of two different back of watches that are brushed. Thefield of view is about 0.4×0.6 mm for a dpi of 10,837. 271 is thebrushed structure of one watch, 272 is the brushed structure of theother one. Picture has been processed to enhance the contrast.

FIG. 28: Picture of cross-correlation of brushed watches. The templateis a picture from the same watch than 271. 281 is the cross-correlationof 281 with the template and 282 is the cross-correlation of 282 withthe template.

FIG. 29: Picture of two different coated dials of watches. 291 is themicrostructures of the coated dial of one watch and 292 of the other.Picture has been processed to enhance contrast. The field of view isabout 1×1 mm and the dpi is 10,837.

FIG. 30: Picture of three different caps. 301 and 302 comes from thesame mould whereas 303 comes from another mould. Picture has beenprocessed to enhance contrast. The field of view is about 5×5 mm and thedpi is 2,400.

FIG. 31: Pictures of three different labels. 311 and 312 comes from thesame printer and the same position on the offset sheet. 313 comes fromanother printer. Pictures have been processed to enhance contrast. Thefield of view is about 2.5×2.5 mm and the dpi is 2,400.

FIG. 32: Pictures of uniform microstructures and its Fourier Transform.261 is a picture with uniform microstructures. 262 is its FourierTransform. The field of view is about 2.5×2.5 mm and the dpi is 2,400.

FIG. 33: diagram showing the difference between the stored image 331 andthe acquired image 332. 333 and 335 are a logo or a macroscopic zone on331 and 332 respectively. 334 and 336 are the microscopic zone ofinterest on 331 and 332 respectively.

FIG. 34: schema showing the different steps of image detection. 341 isthe location and rotation retrieval, 342 is the masking operation, 343is the image pre-processing and 344 is the image-matching.

FIG. 35: Pictures of uniform microstructures. 351 and 353 are pictureswithout flattening. 352 and 354 are pictures with flattening.Cross-correlations of the each image with the template (which matchesimage 354) are shown on the right.

FIG. 36: Pictures showing the effect of different flattening methods onthe cross-correlation. 361 shows the cross-correlation of two picturestaken with a device with specular light where the flattening is done bysubtraction. The SNR of 15.26 dB is better than the SNR of 13.81 dB of362. 362 shows the cross-correlation of two pictures taken with a devicewith specular light where the flattening is done by absolute difference.363 shows the cross-correlation of two pictures taken with a device withdiffuse light where the flattening is done by subtraction. The SNR of10.51 dB is less good than the one of 18.03 dB of 364. This is becausein 363 the anti-correlation is visible and decreases drastically theSNR. 364 shows the cross-correlation of two pictures taken with a devicewith diffuse light where the flattening is done by absolute difference.

FIG. 37: 1D signal illustrating the effect of periodic padding. 371 isthe point where the padding begins. It can be seen that if the firstvalue of the signal is far from the last value of the signal then thepadding induces a border effect

FIG. 38: 1D signal illustrating the effect of mirror padding. 381 is thepoint where the padding begins. It induces less border effect thanperiodic padding. But if the signal is varying a lot at the end, then itinduces some border effects.

FIG. 39: 1D signal illustrating the effect of “Flip Flat”. 391 is thepoint where the padding begins. It induces no border effect.

FIG. 40: Diagram describing the detection strategy progressivelyincreasing cross-correlation sizes. The first Set S₀ contains X₀candidates of size 2^(n). The candidates that have an SNR which issuperior to t₁ are classified in X₁₂. Those which have an SNR which isinferior to t₁ are classified in X₂₂. The set S₁ contains the X₁₂candidates of size 2^(n+1). The same matching is performed at each step.The last set S_(x) should contain only one candidate.

FIG. 41: Pictures depicting the importance of padding disturbingelements to the mean value. 411 is the template, without padding. 412 isanother picture matching with the template, without padding. 413 is apicture from another item, without padding. 414 is the cross-correlationbetween the 411 and 412. 415 is the cross-correlation between 411 and413. 416 is the image 411 padded. 417 is 412 padded. Image 418 is theimage 413 after padding. 419 is the cross-correlation between 416 and417. 4110 is the cross-correlation between 416 and 418.

FIG. 42: Pictures illustrating the effect of padding with smoothborders. 421 shows a picture of a watch called watch 1, which is notwell padded. 422 shows a picture of watch 2, which is not well padded.423 is the correlation between 421 and 422 and it induces a fakepositive. 424 shows a first picture of watch 1 which is padded byputting all the pixels above 20% of the mean value and all the pixelsbelow 20% of the mean value to the mean value. 425 shows a picture ofwatch 2 with the same padding. 426 is the correlation between 424 and425. The correlation is as expected, a true negative. 427 is the samepicture as 424. 428 is a second picture of watch 1 with the same paddingmethod. 429 is the correlation between 427 and 428. It is as expected, atrue positive.

FIG. 43: Scheme representing the transform between to spaces. 431 is acircular donut around a point of interest. 432 is the donut in thetransformed space.

FIG. 44: Removing correlation due to noise. 441 shows the normalizednoise correlation and 442 is the normalized correlation between thesnapshot and its corresponding template. It can be seen that there aretwo peaks of correlation; the sharp centered one (445) is due to thenoise and has a value of λ and the other one is the real peak. It has amean of μ (446). 443 represent the noise correlation, which is stretchedin order to correspond to the noise in the correlation between thesnapshot and the template. It has the peak at the same value λ (447) andthe same mean μ (448). 444 is the normalized subtraction from thecorrelation between the snapshot and the template.

FIG. 45: Difference of Fourier spectrum between images used forFingerprint and random noise images. 451 shows the spatial image usedfor Fingerprint authentication. 452 shows its Fourier transform. 453shows the spatial image of random noise. 454 shows its Fouriertransform.

FIG. 46: Resemblance of Fourier spectrum between images used forFingerprint and random noise images. 461 shows the spatial flattenedimage used for Fingerprint authentication. 462 shows its Fouriertransform. 463 shows the spatial image of random noise. 464 shows itsFourier transform.

FIG. 47: convolution in the spatial domain. 471 shows the filter and 472the picture to be convolved with it.

FIG. 48 shows the convolution of padded image. 481 shows the filter, 482shows the padding and 483 shows the picture to be convolved with thefilter.

FIG. 49 shows the way Fourier coefficients (complex values) can bestored in the database. The coefficients displayed in black are storedin each column 491 of the database table 493. The figure shows thatcolumn 1 has only one coefficient (the average value of the image), thecolumn 2 has the 3 following coefficients, the column 3 has 12coefficients, etc. . . . . This approach enables to optimize therequired bandwidth for transferring data (492) from the database on thehard disk to the CPU. A new line 494 is allocated in the database foreach template image.

FIG. 50 shows the coverage of the database size using the “Best Rank”method. For each set of images of a given size, a certain number C_(ixp)of items should be correlated. C_(ixp) follows a geometrical law. Duringthe detection process, the common ratio of this law is increased untilC_(ix1) is bigger than Card(S₀).

FIG. 51 describes an imaging device featuring both high dpi and largefield of view. 511 is the main case, which guides part 517. This part ismoved in one direction using electric motor 518 connected with to thepart 517 with the threaded rod 513. Likewise, the part 515 is guided bycase 517 and its displacement controlled thanks to electric motor 512using a threaded rod 516. The digital camera 5112 is mounted on part515, along with a lighting system 5113 and a semi-transparent mirror5112 which is used to light the imaged surface with a specular lighting.Alternatively, other types of lighting can easily be conceived in orderto enable diffuse lighting, as for instance the flash of a digitalcamera. The sample to be detected is put on the non-reflective glass519. The motors and camera (and possible lighting) are controlled by acomputer. Optionally, the detection process can be launched using button5110 and result of detection displayed on system 5111 (led or LCD systemfor instance).

FIG. 52: Tree representing the different scale factors or rotationfactors for an item. 521 shows that at low resolution, the templateswith different scale factors have the same Fourier transform. Then, whenthe resolution increases, the different scale factors become distinct.At high resolution (522), there is one Fourier transform per scalefactor. This approach is also true for changes in rotation factor.

FIG. 53: Figure illustrating how a 1D signal can be sampled along acircle 531 of the Fourier transform modulus 532 centered on the DCcomponent 533.

DETAILED DESCRIPTION OF THE INVENTION

Microstructure Images

Image Acquisition Device

Generalities

An image of the surface is acquired by means of a digital imagingdevice, which can be a digital scanner, a digital camera, a mobile phonewith integrated camera (possibly using a special macroscopic lens), aportable microscope, a microscope, a specially designed device, etc. . .. . Such device typically outputs a matrix of values corresponding to acolor component sampled along a uniform orthogonal grid. The imageacquisition can be done by reflection or transparence, depending on theproduct (paper, carton, metal, coating or polymer for instance) and theparticular properties of the material. In the case of imaging byreflection, different types of imperfections may be imaged depending onthe position of the lighting source. The different lighting orientationscan be used to further increase the detection rate (successful findingof a match with a recorded image) and decrease the false positivedetections (wrongly identifying a counterfeit product with an image ofthe database). Different light wavelength may also be used to acquireseveral pictures of the same area (with same or different wavelength forlighting and imaging). In particular objects may be images in differentcolor spaces which are appropriate to highlight the microstructure (RGB,HSV, Lab, etc). This application focuses specifically on the processingof 1 color, but all described processes can also be applied to the caseof object images acquired with several color components. More generally,additional imaging conditions may be chosen for a given imaged productif a risk of false positive/negative is identified based on the imagesrecorded in the database for the previous products. A diagramillustrating this process is shown in FIG. 12. Similarly, it is alsopossible to systematically acquire several images (or other parameters)for a given area and remove, a posteriori, images from the database ifthey are not required to avoid false positive or false negativedetections. The acquisition device should be chosen such that relevantdetails can be seen.

2D CCD with Diffuse Reflection

The diagram for acquisition by diffuse reflection with 2D CCD can beseen in FIG. 17. The image acquisition device (171) can be anything buta digital camera is a good candidate. The item to verify (174) is put ina light box. The light box (173) is used to diffuse the light (172). Itcan also be modeled by an annular flash. This can reveal someinteresting details.

1D CCD with Diffuse Reflection

In FIG. 19, the diagram for acquisition by diffuse reflection with 1DCCD can be seen. The acquisition device can be anything but a scanner isa good candidate. The item to verify (191) is put on a sheet of glass(192). A moving light (193) and a moving mirror (194) are workingtogether to reflect the light on a fixed mirror (195), which reflects iton a 1D CCD. This reveals other interesting details.

2D CCD with Specular Reflection

In FIG. 18, a diagram for acquisition by specular reflection can beseen. The acquisition device (181) can be anything but a microscope(either optical or electronic) is a good candidate.

1D CCD with Specular Reflection

Another possibility consists in modifying a digital scanner as in FIG.18, by moving the CCD or the lighting part. The light (182) is comingfrom a known direction and is reflected (183) by the item to verify(184) in another direction that can be computed with the Descartes Law.The Descartes Law insures that the incidence angle is the same than thereflected angle.

Choice of the Best Device

It should be mentioned that an acquisition device may be capable toproduce images needed for rotation/translation synchronization inaddition to the images used for fingerprint. As it is shown later in theapplication rotation/translation synchronizations are typicallyperformed using images of the object that include specific featurepoints. Such feature points can be acquired using imaging devices withlarge field of views or by using motored multi-axis stages enabling todrive the sample at precise locations or by moving the camera of theacquisition device.

Nowadays such devices exist but they have only a 1D CCD that is movable.It can therefore take two pictures of different resolutions of the sameitem. Unfortunately, such devices are quite slow, especially whenacquiring images at high resolutions. It is possible to specially designa system using a 2D CCD in order to have the same functionalities. Thisdevice features both high dpi and large field of view. It offers thepossibility to put the item to verify at any location on the device andmove the CCD to take pictures from different zones at differentresolutions. The first picture can serve for a macroscopic correlationwhich can be used to retrieve the model type, translation and rotationof the object. After such information has been retrieved, the camera canthen be moved automatically to take the second picture in a specificzone which serves for unique identification or traceability. In FIG. 51,511 is the main case, which guides part 517. This part is moved in onedirection using electric motor 518 connected with to the part 517 withthe threaded rod 513. Likewise, the part 515 is guided by case 517 andits displacement controlled thanks to electric motor 512 using athreaded rod 516. The digital camera 5112 is mounted on part 515, alongwith a lighting system 5113 and a semi-transparent mirror 5112 which isused to light the imaged surface with a specular lighting.Alternatively, other types of lighting can easily be conceived in orderto enable diffuse lighting, as for instance the flash of a digitalcamera. The sample to be detected is put on the non-reflective glass519. The motors and camera (and possible lighting) are controlled by acomputer. Optionally, the detection process can be launched using button5110 and result of detection displayed on system 5111 (led or LCD systemfor instance).

Furthermore the illumination can be either diffuse or specular. In caseof specular light, it will be reflected by some mirrors. The specularlight can reveal different details than the diffuse light. In FIG. 21and FIG. 23, the details of a coated dial of two different watches areshown with specular axial light at about 10,000 dpi. In FIG. 22 and FIG.24, the details of the same watches are shown with diffuse light with 1DCCD (scanner) at 4,800 dpi. It can be seen in these figures that for thefirst watch it is better to use diffuse light as there are more detailsin FIG. 22 than in FIG. 24. On the contrary, for the second watch it isbetter to use specular light as there are more details in FIG. 23 thanin FIG. 24.

The above described device is generic as described in FIG. 20. It canhave a specially designed excrescence (202) on which it is possible toput any item. For example, a watch with a strap that cannot be openedcompletely can be slide on the excrescence.

Characteristics of Fingerprint Noise

Ideal Case Versus Real Case

The ideal case for fingerprint identification is an image made out ofpure white noise. Indeed, in such case all frequencies contributeequally to the correlation, which maximizes the detection robustness andselectivity (i.e. difference between true positive and true negativedetections). In practice, real image statistics of object are differentfrom white noise. It is important to notice that if the image differstoo much from noise, it wouldn't be possible to recognize it bycross-correlation. In fact in this case macroscopic details will mainlycontribute to the cross-correlation and then picture from differentitems which have a reassembling macroscopic structure will be consideredas being the same. For these reasons, it is important to characterizethe deviations in order to remove them. Some of these deviations can becharacterized easily in spatial domain (lack of homogeneity forinstance) and others can be more easily detect in the frequency domain(anisotropy and spectrum shape). In FIG. 45, it can be seen that theFourier spectrum (452) of a spatial image used for Fingerprint (451) isquite different from the Fourier spectrum (454) of random noise (453).The following describe some approaches to correct such deviations:

Change of Resolution

Microstructures have typically different types of defects and each onecan be ideally imaged at a specific resolution. For instance, let usconsider the case of circular defects made by uniform circular disks of10 um of diameter uniformly spread on the sample such that 50% of thesurface is covered. A resolution of 2400 dpi will result in each defectbeing resolved by 1 pixel, a resolution of 24,000 dpi will image onlyone defect on a 128×128 image and a resolution of 240 dpi will averageof approximately 100 defects for each pixel. In such a case, it can beforeseen that 2400 dpi will be the best resolution, and that theobtained image will be closest from white noise.

Pre-Processing

It is also possible to alter the image by processing it in order to haveits statistics get closer to white noise. One possible pre-processing isflattening and is described extensively below, such approach will inparticular decrease lack of homogeneity typically caused by non-uniformobject lighting. FIG. 46 shows that the Fourier spectrum (462) of theflattened image (461) is very like the Fourier spectrum (464) of randomnoise (463). Spectrum whitening is also a possibility of pre-processingto make the picture very like random noise. Spectrum whitening isdescribed extensively below.

Post-Processing

It is also possible to process the Fourier coefficients of the images tobe correlated. The interest of such approach is that it enables to useonly those coefficients which describe the noise of the image. Forinstance, macroscopic variations can be partly cancelled by not usinglow frequency coefficients; anisotropy variations can also be decreasedby threshold or morphological filter of the spectrum.

Resolutions Requirements

It is basically possible to use any sufficiently random structure touniquely identify a product. In particular it is possible to use themicrostructure of a graphical design (in particular the barcode, text,logo, artistic designs, lines, etc) printed on a product as a randomidentifying structure (use of barcode has also been described forcounterfeit detection in U.S. Pat. No. 6,869,015, using a system basedon particles mixed with the barcode) or to use the microstructure of theunderlying material. However, the resolution of the imaging deviceshould be adapted to the characteristics of the microstructure.

An example of such a microstructure is given in FIG. 2. It shows thetypical microstructure (the word “imperfection” is also used with anequivalent meaning in this document) of regular paper as seen with anequivalent resolution of about 11,000 dpi, which corresponds to aresolution of a few micrometers. In practice, it is considered that forstandard paper and carton, the random structures have a size that rarelyextends above a few millimeters (however, some special material may havemuch larger random structures or highly anisotropic structure sizes).Typically, the considered structure size will be below 1 mm or evenbelow 100 um. Standard imaging devices listed above can reach such levelof details. For instance a 1200 dpi scanner can theoretically resolvestructures of 21 □m. Other type of materials, like polymer, metal,coating or glass for instance, may exhibit different types ofmicrostructures requiring either lower resolution (and thus larger fieldof view in order to keep the same quantity of image information) orhigher resolution. In FIG. 3, the microstructure of a metallic part isshown at 4,800 dpi. The size of the microstructure elements are about 47um. The field of view is about 10 mm×10 mm. In FIG. 4, themicrostructure of a piece of polymer is shown at 2,400 dpi. The size ofthe defaults is about 349 um and the field of view is 50 mm×50 mm. InFIG. 5 the microstructure of an offset printed label is shown at 2,400dpi. The size of the defaults is about 42 um and the field of view is2.5 mm×2.5 mm. In FIG. 6 the coated part of a dial is shown at 10,837dpi. The size of the details is about 58 um and the field of view is 1.2mm×1.2 mm. For all these materials the details are uniformly spreadacross the image. Thanks to some image processing steps that aredescribed in more details in the next sections, it is possible to obtaina uniform variation of the microstructures inside an image. Furthermoreit was noticed that these microstructures were randomly spread on theitem. This insures randomness between different items.

Generally speaking, it appears that a resolution of 2400 dpi or higheris often required to characterize a unique fingerprint. However, somematerials exhibit unique fingerprints with significantly lowerresolutions, this is particularly common for textures which have beendesigned, on purpose, with a noisy and variable pattern. Furthermore,unique fingerprint can be used in order to perform 1 to 1 matching. Inthis case the resolution can be lower because it is not necessary tocompare with all the items in the database but only with the claimedone. 1 to 1 matching can be performed in particular when a serial numberis displayed on the item to verify. This serial number can be read by ahuman operator or via OCR and transmitted to the matching algorithmwhich will then only compare with the claimed item. In the case of amicroscopic fingerprint, a lower resolution may be sufficient. Finally,macroscopic fingerprints can be performed with resolution of 300 dpi orlower. FIG. 25 is the schema representing the different dpi necessaryfor each level of the fingerprint. The different technologies arerepresented on the vertical axis (251) and the different dpi's arerepresented on the horizontal axis (252). There are always gray zonesand white zones for each technology. The white zone represents the bestdpi zone for the technology whereas the gray zone shows up to where itis possible to extend it. Unique fingerprint (255) has no upper boundaryas the dpi to identify an item uniquely depends on the size of itsmicrostructures. Unique fingerprint may also extend in some cases in thelower resolutions area (like leather for instance and more generally anynatural material). Microscopic fingerprint (254) and macroscopicfingerprint (253) provide very close functionality. The only differenceis that, in microscopic fingerprint, microstructures are consideredwhereas in macroscopic fingerprint macrostructures are considered. Anupper boundary is therefore necessary because it defines the boundarybetween microscopic and unique and between macroscopic and microscopic.For the same reasons a down boundary is also required.

Microstructure of Printed Paper

A particular type of randomness can be observed in the contours of aprinted shape or inside a tint area or at locations where several colorsare applied. For instance FIG. 13 shows the contours of text printed inrotogravure technology and how they vary between two successive prints.In this case, these contours have some features which are common betweentwo contours (like 131 and 133 for instance) but also features which aredifferent (like 132 and 134). Common features (i.e. micro fingerprint)typically have sizes that extend on a larger area than differentfeatures (i.e. unique fingerprint). This means in particular thatcontour matching can be performed using high frequencies of the signal.The similar large features come from cells location on the cylinder andthe variation between two prints comes from the texture of the paper andhow the ink deposits on the surface.

More generally, the print randomness depends on several parameters,including the location of the cells engraved on the cylinder, theviscosity of the ink, the printing speed and the micro structure of thecarton, the ink filling of the cylinder cells, etc. on one hand and theink absorption on the paper on the other hand. All these parameters havean influence on the randomization of the shape contours or tint area. Itis this randomness that can be used to identify individual products butalso production batches, individual cylinder used for production or theprinting works. It is also possible to deduce the aging of the cylinder(and hence at which approximate time a product was printed) from theimages (for instance, the doctor blade that pushes the ink in arotogravure cylinder progressively wears the engraved cells aftermillions of rotations). In particular, the common features of the 2images of FIG. 13 come from the fact that they have been printed withthe same cylinder; another cylinder engraved with slightly differentparameters would provide substantially different contours. All thesetechniques are not limited to rotogravure and also applied to anyprinting technology. Another example is shown in FIG. 14 with a tintarea printed with industrial offset (the original image was yellow andwas later processed to enhance the contrast). The image is clearly notuniform and will always differ between two different prints.

Random Microstructure of Other Materials

Microstructure of Chemical Etching

In order to illustrate this type of structure with a practical example,we consider in the following the particular case of a marked watch case.Such engraving are typically performed by chemical etching. So eachengraved logo is unique as it can be seen in FIG. 15. 151 is a part of achemically engraved logo of a back of a watch and 152 is the same partof the same logo on another watch of the same model. This is an exampleof unique fingerprint. Chemical etching is usually done on metallicsurfaces but can also be applied to any other material.

Microstructure of Stamped Material

The back of the watch can also be stamped instead of chemicallyengraved. For this kind of material, the dpi of the acquisition deviceshould be much higher than in the preceding case. If not, it will onlybe possible to distinguish the shape of the stamp used to mark thesurface. It is shown in FIG. 26 that with a big enough dpi, it ispossible to see differences between the two items. Here the field ofview is about 0.4×0.6 mm and the dpi is 10,837. This is an example ofunique fingerprint. Stamping can also be used for microscopicfingerprint. In fact, if the stamp is marked with random noise, it willalways display the same pattern on the object where it is applied. Inparticular, this technique can also be used in the case of pillsmanufacturing: Indeed in such case, pills are typically made of powderwhich is compressed in order to create a solid object. The surface ofthe tool used to compress the powder can be marked with noise, in such away that each pill features the same random pattern (i.e. microscopicfingerprint). Again, stamping is usually done on metallic material butit can also be done on any other material.

Microstructure of Brushed Material

Another standard surface finishing is brushing, typically on metallicsurfaces (like watch case for instance) but brushing finish may alsoapply for many other materials. Again the ratio between the dpi and thefield of view has to be high enough to see details. In FIG. 27 it isdifficult to see the details with naked eyes because the brushing isvery present. But the microstructures are here. It is shown in FIG. 28that the cross-correlation is not damaged by the brushing. Thecorrelation peak is a bit stretched out but as there is no peak when twodifferent items are compared, this is not a problem. This is an exampleof unique fingerprint. Brushing is often done on metallic surfaces butcan be applied to any other material.

Microstructure of Laser Engraved Metals

Laser is the English acronym for “Light Amplification by StimulatedEmission of Radiation”. Different types of laser exist but the mostcurrent one is the CO2 laser. Laser engraving is often used for littleseries or for unique marking. For example, serial numbers are very oftenlaser engraved. This gives the possibility to recognize the number withOCR and then perform a 1 to 1 matching with the claimed number. If theengraving is not unique, laser engraving is one example of uniquefingerprint by 1 to n matching. In fact, the laser burns the metalswhere it is applied. So the microstructures of the metal becomeapparent. FIG. 8 shows the microstructure of laser engraving at 30,000dpi with a field of view of 0.13 mm×0.13 mm. Of course laser engravingcan be done on any material, such as plastic, and not only on differentmetals. A particular attention has to be given to the fact that laserengraving induces some pulses that can be perceived as repetitive if theresolution of the taken picture is not high enough. Such pulses are notpresent at the beginning and the ending of the engraving. In FIG. 9, 91shows the structure of the beginning of the engraving and 92 shows thestructure of the pulses. Therefore if the resolution of the acquisitiondevice is quite low, for example 6,400 dpi, the matching can beperformed in the beginning or in the ending of the engraving to avoidthe periodicity of the pulses. On the contrary, when the resolution isbecoming big, for example 30,000 dpi, these special zones should beavoided because the details are too big and the image is not noisyenough. In this case, the repetitive pattern of the pulses is notvisible and the matching should be done into the pulses area.

Microstructure of Homogenous Metals

It is possible to detect the microstructure of a homogenous metal partat a quite low dpi. FIG. 10 shows that a 7.1 Mega pixels digital camerawith an optic of 28.8 mm and a focal distance of 20 centimeters isenough to see these microstructures. The resulting picture is at 1445dpi and the field of view is 2.2 mm×2.2 mm.

Microstructure of Resins

Solidified resin items can be identified with the unique fingerprinttechnology as their microstructure is unique. FIG. 7, shows themicrostructure of resin at 20,000 dpi. The field of view is 0.35 mm×0.35mm.

Microstructure of Coating

Different types of coating exist; in particular coating can be used forprotection or finish purposes others can be used for coloring purposes.Application means are also various, but typical methods include bath,electro-coating, pulverization or serigraphy. FIG. 29 shows thedifference of microstructures of a painted coating at about 10,000 dpiwith a 1×1 mm field of view. Coating can be put randomly and it thiscase it is a very good candidate for unique fingerprint authentication;but it can also be applied on purpose at some specific random locations,which make it a very good candidate for microscopic fingerprintauthentication.

Microstructure of Molded Materials with a Non-Glossy Surface

If a material is molded, (particularly if it is molded with a sandedmould which creates “sanded finish” appearance), the microstructures ofeach molded part look similar. So if the dpi is chosen carefully, allthe items that come out of a same mould feature similar surface, whereasthose who come out of some different moulds look completely different.This is illustrated is FIG. 30. 301 and 302 come out of the same mouldand have the same microstructures whereas 303 comes out of another mouldand has other microstructures. Pictures are taken at 2,400 dpi with afield of view of 5×5 mm. An example of that is the case of cover of somecontainers. This is a typical example of microscopic fingerprint.

Microstructure of Molded Materials with a Glossy Surface

Molded materials with a glossy surface are not very good candidates forthe unique fingerprint technology because it is necessary to use veryhigh resolution to model the differences between items. But if themolded material is a plastic and if it contains fiber glass, it becomesa very good candidate for the unique fingerprint technology because thefiber glass induces random patterns. It is possible to resolve detailsat a quite low resolution. FIG. 11 shows a smooth molded plastic partwith fiber glass at 1445 dpi. The field of view is 2.2 mm×2.2 mm.

Microstructure of Ink Deposit on Non-Porous Materials

If ink is deposited on a non porous material, for example glazed paper,each drop has its own shape and location. Labels are taken as anexample. A label will be the same than all the other labels that areprinted with the same printer on the same position of the offset sheet.On the contrary, a label printed by another printer or in anotherposition of the offset sheet will look different. This is shown in FIG.31. 311 and 312 are printed with the same printer and the same positionon the offset sheet. 313 is printed with another printer. This is againan example of microscopic fingerprint.

At the extreme, this technology can be extended by reducing the dpi inorder to check if the overall graphical design of a box matches with theoriginal design. This approach is for instance applicable to fakemedicines that are sold on the Internet. This is a particular example ofmacroscopic fingerprint.

Other Microstructures

More generally any surface treatment or machining process resulting innoisy images or images with sufficient entropy may be used as a templatefor image retrieval. This includes in particular surface sanding,milling, spinning, drilling, etc. . . . .

Image Recording

General Strategy

First of all, a specific area has to be defined; the same area will beused for recording and future comparisons. The location of this area isoptimal when its microstructures are uniformly spread out the image asit was explained above. Such an area can be retrieved automatically.Indeed, this area should match some specific mathematical requirementsenabling to maximize detection rate and minimize false detections. Inparticular, images having statistics of uniform noise have proven toperform particularly well. Many approaches are available in theliterature which enable the measurement of noise characteristics and maybe used here. One example is to analyze the Fourier transform of theimage, for instance white noise is known for having a flat spectrum. Itis also possible to derive the optimal dpi resolution by analyzing withpart of the Fourier spectrum characterizes a noise signal as describedabove. This is shown in FIG. 32. 322 is the uniform Fourier Transform of321 which contains microstructures that are uniformly spread out overthe image. It is possible to take one or two images at two differentlevels of details for pre-processing steps purpose. This depends on thefuture processing and can be determined by the material, the shape andthe application.

Storage Issue

Type of Storage

The data can be stored in different ways. When using the macroscopicfingerprint technology or the microscopic fingerprint technology, theamount of data to store is much smaller than the amount of data of thenumber of items to protect. For this reason, if the amount of data issmall it is possible to store it together with an ID (which is a uniqueidentifier of the item) in a binary file which is stored somewhere onthe computer. When using the unique fingerprint technology, on thecontrary, the amount of data to store is the same than the amount ofdata of the items to protect. This can become huge, especially if thereare lots of items to protect (here millions or billions). In this caseit is no more possible to store data in a binary file. It is thereforenecessary to store data in a place where it can easily be accessed. Adatabase appears to be the most flexible storage place. It isself-contained, can be backupped and moved from one place to the other.Furthermore, it can be combined with an existing database at thecustomer place, for example SAP.

Location of the Database

The database can be either local or remote. It can be accessed via acentral server which can also be remote. The connection can be doneacross any network such as the internet or the GSM network.

Image Compression

Once the microstructure has been digitized, the corresponding image hasto be stored and will constitute the record set (typically accessedthrough database tools). The basic requirement is to obtain an image ofthe microstructures that contains a sufficient amount of details inorder to enable a unique identification (knowing that the total amountdata will also increase with the database size). This amount of detailsis of course directly linked to the image size and to the microstructureelement sizes. A theoretical boundary of the minimum image size for agiven database size may be computed in the particular case of black andwhite image. Indeed, for a 5×5 pixels image, the number of differentpossible images would be 2^25˜33 millions. This means that an idealrandom black and white structure could be uniquely identified in adatabase of 33 millions reference images. In practice much larger images(up to a size of 100 times more in each dimension) have to be used anddatabase size may become an issue.

Various lossy compression schemes may be used to decrease the volume ofthe image database. Indeed, even though such compression alters theimage, experiments show that this image matching can be performedsuccessfully even for high compression ratios (compression may beperformed on both the reference and the tested image or only on thereference image). For instance, many experiments have shown thatcompression ratios of 50 to 100 can be applied using for instancecompression scheme based on the DCT (Discrete Cosine Transform).Therefore, a 100 by 100 pixels grayscale image, which represents 10kilobytes, can be compressed down to 0.1 kilobyte. For storing scans of10 millions of items, hard disk capacity of 1 gigabyte is thereforesufficient (similarly 1000 by 1000 pixels images would require 100gigabyte of space, which is really reasonable). The use of waveletcompression may also be useful, as it would benefit to the imageretrieval task. Indeed the image retrieval can then be hierarchicallyperformed on increasing wavelets frequencies of the images (thusspeeding up the retrieval process for instance). Other information mayalso be stored in place or in complement of the scanned images. Forinstance, invariant features (image moments for example, statisticalproperties, etc), parameters related to contours, minutiae features orany other information able to speed up the retrieval process or increasethe matching reliability.

Storing Multiresolution Images

In the case when multiresolution images have to be stored, it ispossible to store them efficiently, in order to avoid storing redundantinformation. For example, if the correlation is done in the Fourierdomain, it is interesting to store only the Fourier coefficients of theimage as well as the ID. Since a scale in the spatial domain induces acrop in the Fourier domain, this property can be used to improve theefficiency of the storage. In fact it is possible to store only thecoefficients of the highest resolution image, split across the differentcolumns of the database so they can be accessed easily. The coefficientscorresponding to those of the lowest resolution image can be stored inone column, then the remaining coefficients corresponding to thecoefficients of the next resolution image minus the coefficients of thelowest resolution image in the next column, and so on. An example isgiven for storing Fourier transform of images in the paragraphconcerning Decision Tree and in FIG. 49.

Database Efficiency

In addition to the type of information stored (Spatial data, Fourierdata, Wavelet date, etc. . . . ) which has been described above, thesoftware architecture used for storing data is also critical. Indeed itcan impact in many ways the speed and the storage size. In order toincrease the efficiency of database access, it can be partitioned eitherhorizontally or vertically depending on the type of access. Wheninserting a huge number of rows in the database it can be noticed thatthe time for retrieval is not linear. This is because the database isbecoming too large to be accessed in a reasonable time. Partitioning isbetter than making views because it is completely transparent to theuser. Furthermore, as views are dynamic virtual tables, changing thedata in the table alters the view and therefore induces an additionalcomputation time. What's more, the view should be created by the userand is not part of the design of the database.

-   -   Vertical partitioning consists in storing some columns of the        table in a specific location and the other columns in another        location. This is useful if some accesses are done to retrieve        only part of the columns and other accesses to retrieve another        part of the column. The columns that are often accessed together        are stored at a same location.    -   Horizontal partitioning consists in storing some rows of the        table in a specific location and the other rows in another        location. This is useful if only some parts of the rows are        accessed together. For example “n” threads can access each a        subset of the rows. The rows that are accessed together are        stored in a same location.

The different partitions can be stored in the same file or in differentfiles. They can also be stored on different hard drives. In fact, ifthere are too many concurrent accesses to the database, the maximumtransfer bandwidth of the hard drive is reached. Therefore, using morehard drives can speed up the I/O operations.

Location of Snapshot

It should be noted that in the case of brand protection, it might beparticularly interesting to exploit the microstructure of areas that areclose to the logo (or any brand identification means). Indeed, parallelimporters often try to destroy traceability information (like UV codesor datamatrix). They typically achieve this by scratching the surface orpainting over the code. Surprisingly, doing so is often not illegal.However, altering the logo is illegal (moreover it clearly alters theproduct merchantability). Therefore, using microstructure of the logo orthe underlying medium may help to protect from pirates alterations.Finally, the logo is a simple identifiable feature, which makes itconvenient for inspecting purposes (in case the inspection processexplicitly requires imaging a given area of an item). This idea is notlimited to the logo and also applies to any trademark, copyrightprotected design or, more generally, to any design protected by the law,printed on packages or documents.

Image Detection

General Strategy

The general strategy is depicted in the schema of FIG. 34. When thepicture is taken, its position and rotation in respect to the originalimage is not known. The new coordinates x′ and y′ have to be deducedfrom the translation vector, the angle of rotation and the old x and ycoordinates. This is illustrated in FIG. 33. The rotation can beretrieved by a Fourier transform or by iterative steps. The rotationcompensation strategy is explained in more details below. The locationcan be retrieved by a 1D projection for example. Then, masking (332) hasto be computed, depending on the size of the field of view and on thesize of the microstructures. After that all the pre-processingoperations (333), such as flattening can be computed. Finally, the imagematching (334) is performed. After image matching, a metric enables todecide that one or several images from the record set do actually matchto a certain level. Ultimately, the number of images belonging to thismatching set should be 1 (cardinality equal to singleton). Depending onthe application, this image will identify a unique object (and anyuseful side information, typically by using a database access) or set ofobjects with common features. For instance, in the case of microfingerprint applications, such common feature can be a mould identifier.Another example, for macro fingerprint applications, may be a genuineprinted package.

Rotation Compensation

The rotation angle can be found if the picture includes some anisotropicvisual information which is identical for all items. For instance, ifthere is a line across the picture, it will be possible to use thisinformation in order to exactly find the rotation angle. Of course thisline will be a drawback for cross-correlation purposes. That is why itis possible to take different pictures from the item to verify, even atdifferent resolutions and with different fields of view. Each image canthen be assigned to a specific operation. In practice it is possible tocapture an image with a higher field of view of each template, with thepicture containing the region of interest or not. Then the rotationangle of the template is retrieved. The same operation is applied to thesnapshot image. The compensation angle is then defined by the differencebetween the two rotation angles. Transform domains may be used to aidthe extraction and characterization of anisotropic information. Forinstance, the Hough transform or the Fourier transform may be used. Inthe case of the Fourier transform, the spatial anisotropy of the imagewill result in a line in the modulus of the Fourier transform of thisimage. This is shown in FIG. 16. 161 shows the Fourier transform and 162shows the angle, which is a radial sum of the value of the FourierTransform related to each angle. One solution for finding the mainanisotropic direction is to compute the following equation:p(θ)=∫₀ ^(R) l(r,θ)·drwhere l(r,theta) is the value of the modulus of the Fourier transform atthe polar coordinates r and theta, l( ) the modulus of the Fouriertransform, and p( ) is the value of the accumulated modules for a givenangle (162).

This angle is actually determined with a 180° accuracy because ofFourier transform symmetry properties of real images. Moreover, thespatial anisotropy of the image may contain several main directions (forinstance a square contains two main direction which are perpendicular)which results in different candidate angles. The right angle can bedetermined by testing each of the possible angles. When the spatialanisotropy contains only one main direction, this direction isperpendicular to the angle found by the sum over the axes. That is why,in FIG. 16, the sum is done over an angle a in 161 and the result isdisplayed for an angle “a+90”. A testing procedure consists incross-correlation with a reference image.

It should be noted that the image used for rotation detection (and thecorresponding reference image) may be taken at a resolution (dpi) and ata location that is different from the image used to characterize themicrostructure of a surface. Finally the angle value can be moreprecisely determined by a trial error procedure around the anglepreviously found. For instance, it is possible to iterate between −1 to+1 degrees by steps 0.1 degrees, computing the SNR for each angle, andkeep the angle that results in the best SNR.

Once the rotation angle is known, it is easy to retrieve the location ofthe specific pattern to cross-correlate. In fact, it could be possibleto store in the database the coordinates and size of the patternrelative to a certain angle. In FIG. 33, x and y are stored in thedatabase. X′ and y′ can be retrieved easily once the displacement androtation is known.

It is also possible to adopt a totally different strategy that avoidscomputing the rotation angle: the basic idea consists in taking acircular shape around a point of interest and unwarping to a rectangularshape. FIG. 43 describes this process. 431 is the circular donut arounda point of interest that has to be cut. The cut is done in AB. A is theorigin and B is the end of the new image. The circular shape is thenmapped (or unwarped) on a rectangular (432) shape using standard imageinterpolation methods. Such image can be compared (for instance usingcross-correlation) with a template image that has been generated usingthe same process. The location of cross-correlation peak will vary withthe rotation angle, but the SNR value will not be affected. Thereforethis technique enables to detect matching images without the need for aprior compensation in rotation. The technique can also be used using a1-dimensional signal: in such case, the circular shape is a circle,along which the image is sampled.

Another possibility to avoid computing the rotation angle is totransform the (x, y) coordinates of the image into log-polar coordinates(ξ, η). The mapping is done as in Equation 1.

$\begin{matrix}{{\xi = {\log\sqrt{x^{2} + y^{2}}}}{\eta = {\arctan( \frac{y}{x} )}}} & {{Equation}\mspace{20mu} 1}\end{matrix}$Translation Compensation

Translation compensation is a critical operation in order toautomatically perform the detection at the exact location where thereference image is located. A general approach to compensate fortranslation is to use an image that includes feature points or a knownfeature shape. The location of the region of interest, with respect tothe feature or set of feature points is known. In practice, one approachis to acquire a low resolution image of the object, with higher field ofview, such that both the region of interest and the feature points orshape are visible. This operation is done for each snapshot and for aset of templates. For the set of templates an image is captured andtransformed into a mask. The mask will be unique for a whole set ofobjects. The snapshot image is then correlated with the template mask.The position of the cross-correlation peak enables to find exactly theX/Y position of the snapshot and therefore to determine the location ofthe region of interest that should be acquired at higher resolution withnarrow field of view. When correlating at high resolution the positionof the correlation peak indicates the translation vector. So a new imagecan be captured in the correct position in order to perform a matchingof perfectly aligned pictures. This operation will drastically increasethe SNR of the good match.

Preprocessing

Generalities

The cross-correlation is typically computed on flattened images. One wayto perform flattening is to do a mathematical operation that consists intaking the difference between an image and its frequency low-passedversion. The goal of this operation is to remove macroscopic colorvariations across the image and which are not significant, like lightingvariations for instance. In particular, it enables to set all theborders of the image to the same value, which avoids border effectsduring a cross-correlation based on Fourier Transform (FFT basedcorrelation implicitly tiles the image, artificially creating bordereffects between tiles if image is not uniform). This operation isnecessary because in practice the captured picture is not a white noise.So the aim of the flattening is to make the captured image very like awhite noise. In FIG. 35, the effect of flattening is shown. 351 and 353are images that have not been flattened. 352 and 354 are the same imagesafter flattening. In this example, the cross-correlations are computedwith a template image that should match with the image 353 but thatshould not match with the image 351. The cross-correlations for eachimage are shown on the right side. It can be seen that thecross-correlation image 355 and 356 show no correlation peak, which isnormal. The image 357 does not show a clear cross-correlation peakalthough the image should normally match with the template. Thecross-correlation of the flattened image 358 shows a clearcross-correlation peak.

Different Methods for Flattening

The flattening can be done with several different methods. The first oneis the absolute difference between an image and a low-passed version ofit. It is described by Equation 2. B is the flattened image, A is theoriginal image and f(A) is the low-passed image or any function of theimage enabling to perform flattening.B=|A−f(A)|  Equation 2

The second one is the subtraction between the image and a low-passedversion of it. It is described by Equation 3.B=A−f(A)  Equation 3

The third one is the blending of the image and the opposite of itslow-passed version g(f(A)). A is weighted with α and g(f(A)) with 1−αfor the blending. α is in [0,1].

Any other method that helps to remove macroscopic color variation orthat highlights the random noise of the image can be used. For example,the Fourier transform can be thresholded in order to correlate only somerelevant coefficients. Another possibility is to perform spectrumwhitening as explained below.

Alternatively the first or the second method gives better results. Infact, the subtraction gives better results when the device used tocapture the image, such as a microscope, has a specular light. If thedevice, such as a digital scanner, has a diffuse light, then theabsolute difference gives better results. Theoretically, the subtractionshould give better results because it really models the differencebetween the two images. Even if no information is lost, the absolutedifference reduces the dynamic range by two; therefore the results ofthe correlation are less good. This can be seen in FIG. 36. 361 has abetter SNR than 362 because the flattening of 361 is done by subtractionwhereas the flattening of 362 is done by absolute difference.Nevertheless, when the device has a non specular light, ananti-correlation (as in 363) appears next to the correlation. Thisanti-correlation is so high that the SNR of the correlation decreases alot. In this case if the absolute difference is computed, theanti-correlation is transformed into a correlation and the SNR issignificantly increased (as in 364).

Different Spaces for Flattening

Some flattening methods imply the calculation of a low-passed version ofthe captured image. This low-pass is often performed by convolving aGaussian filter with the captured image. A Gaussian filter is displayedin Equation 4. The size of the filter is 3σ×3σ. Its origin is in itscenter. The r in the formula below indicates at which radius of thecenter the current pixel is.

$\begin{matrix}{{f(r)} = {\frac{1}{2{\pi\sigma}^{2}}{\mathbb{e}}^{\frac{- r^{2}}{2\sigma^{2}}}}} & {{Equation}\mspace{20mu} 4}\end{matrix}$

The Gaussian filtering can be performed in different mathematicalspaces. Two examples of these spaces are the Spatial Domain and theFourier Domain. It has to be noted that a cross-correlation in theSpatial Domain corresponds to a multiplication in the Fourier domain. Ifthe image becomes large, the cross-correlation is less time consuming inthe Fourier Domain than in the Spatial Domain. It is generally admittedthat if a sequential process is used, the cross-correlation is morerapid in the Fourier domain if the size of the image is bigger than21×21 pixels. Nevertheless the convolution in the Fourier Domain canlead to unexpected results. In fact, the image is repeated periodicallyin order to transform it in the Fourier Domain. If all the borders donot have the same value (which is generally the case and one of the mainreason why flattening is needed), this will introduce discontinuities inthe signal. That's why the result of the convolution can be different aswhat is expected. Some solutions can be used in order to smooth thediscontinuities in the border of the signal. One of these solutions ispadding of the original image even if the convolution is done in theFourier domain.

Different Padding for Flattening

To perform the convolution between the captured image and the Gaussianfilter it is necessary to pad the image. In the spatial domain, this isstraight forward as it can be seen in FIG. 47. To insure all the pixelsof the filter (471) are convolved with the image (472), it is necessaryto pad the image. In the Fourier domain, the padding can reduce thediscontinuities induced by the Fourier Transform. The padding can beperformed with different approaches. FIG. 48 shows the utility ofpadding. The filter (481) can be convolved with the image (483) and allthe pixels of the filter can be used because the image is padded (482).

-   -   Zero padding is the simplest padding. It consists in adding        zeros on the whole surface that should be padded. This is not        very efficient because it induces border effects.    -   Periodic padding consists in repeating the image horizontally        and vertically. This has less border effects than zero padding.        FIG. 37 shows an example of periodic padding of a 1D signal. 371        shows the discontinuity of the signal    -   Mirror padding consists in repeating the flipped image        horizontally and vertically. This has very few border effects.        FIG. 38 shows an example of mirror padding of a 1D signal. 381        shows the slight discontinuity of the signal.    -   A special padding consisting in mirroring the signal, then        inverting it and finally shifting it so that the first value of        the padded part is the same than the last value of the non        padded part. FIG. 39 shows an example of Flip Flat padding for a        1D signal. 391 shows that there is no discontinuity in the        signal.        Adjusting the Sigma and Radius for the Flattening Filter

If a Gaussian filter is chosen, its size can be fixed to 3σ×3σ. In factthe width of the Gaussian depends on the sigma. If the size of thefilter is 3σ×3σ, then the values in the borders of the filter are closeto zero. There is therefore no need to take a bigger size.

The sigma should be chosen carefully in order to give the best results.The sigma should be adapted automatically because pictures of differentitems can have different microstructures of different sizes. Furthermorethe material properties can induce different lighting distortions.Therefore, the sigma needed for flattening could be different from onetype of item to the other. It is then necessary, for industrializationspurposes to have an automatic way to optimize the sigma. There areseveral possibilities listed below to optimize it.

-   -   One possibility is to flatten the image with a given sigma and        perform its auto-correlation to compute the SNR. When the        obtained SNR is maximal, it can be said that the sigma is        optimal. This optimum can be found by any optimization method        such as the dichotomy method.    -   Another possibility is to analyze the Fourier spectrum of the        flattened image. When the spectrum is the most next to white        noise spectrum, the sigma is optimal.    -   Masking of macroscopic details can also be done. This is        explained extensively below.        Masking        Motivation

If the field of view has a size that implies that some macroscopicdetails are visible, they should be masked. In fact macroscopic detailsoften induce a macroscopic correlation which can imply a false positivedetection. The importance of masking disturbing elements is shown inFIG. 41. It is shown that, without masking, the cross-correlation (414)of a template (411) and another picture of the same item (412) is notbetter than the cross-correlation (415) of the template (411) with apicture of another item (413). Even worse, the cross-correlation (415)of the template (411) and the picture of the other item (413) has a goodpeak because of macroscopic cross-correlation. This kind of mismatchingis a false-positive detection that must be absolutely avoided. It isshown that it is possible to prevent such mismatch by padding to themean value the disturbing element.

Masking Fixed Locations

One possibility to do the masking is to put all the values that have notto be taken into account to a fixed value which can be either the meanvalue of the image, or a value, which will not change the mean value ofthe image, or even noise. In FIG. 41, the padding is done with the meanvalue of the pixels that are not cancelled by masking. When the templatepadded (416) is cross-correlated with another picture of the same itempadded (417), the peak of cross-correlation (419) is visible. When thetemplate padded (416) is cross-correlated with a picture of another item(418), there is no more peak (4110) of cross-correlation. This exampleillustrates how a masking approach can help to avoid false-positivedetection and also improve the detection rate. Such mask may be recordedin the database.

Masking with Smooth Borders

This approach may be insufficient in some cases. For example the paddedarea can induce borders that will highlight the macroscopic correlation.In FIG. 42, 421 and 422 are not well padded. Therefore 423 is a falsepositive. If this is the case, it is possible to pad every pixel that isbelow a certain threshold b or above another threshold a in addition tothe previous padding. This padding has to be done with the same value mused for the previous padding.v(x,y)>a=>v(x,y)=mv(x,y)<b=>v(x,y)=m  Equation 5

This has the effect to smooth the borders of the padded area as for 424,425, 427 and 428. Furthermore, two successive snapshots can haveslightly different areas that are padded. With this kind of padding thetrue negative (426) can be differentiated from the true positive (429)and there is no more problem of fake positives.

Overview of Matching Algorithms

The image matching purpose is to define a metric which enables to decideif two images are similar. A perfect similarity can be characterized bya null Mean Square Error between two images and a matching metric willtypically converge to a known value in this case. Image matching can beperformed using cross-correlation or matching in transform domains(Hough or Fourier spaces for instance) possibly enhanced by log-polartransforms in order to be invariant with scale or rotation. It is alsopossible to use spatially transform domain in order to map the image ina way that is more efficient for matching purposes. For instance, it ispossible to compute a 1D profile corresponding to the sum of colorintensity along each line and use such 1D function in conjunction with1D signal matching algorithms for fast retrieval (possibly refined witha vertical profile). Another spatially transform domain was shown inprevious section and illustrated in FIG. 32. The so-called “minutiae”approach used in fingerprint matching can also be used. Its principle isto use only a limited number of localized feature points instead ofusing the whole image.

In the case where microstructure of printed area is used (like in FIG.13), it is also possible to use matching techniques based on contours ofthe printed shape. State of the art contour matching techniques may thenbe used as for instance Fourier descriptors or 1D cross-correlation (thecontour being converted to a 1D signal).

Artificial neural networks may also be used in order to perform theimage matching or the contour matching operation. More generally, letthe image be described by a given set of parameters. Any classifier maybe used to find the best match in a database. The choice of the bestclassifier will depend on the computational requirements, the imagestatistics, etc. In order to help the image matching operation, theknowledge of the original design may also be used. For instance FIG. 13corresponds to a letter which orientation, scale and spatial position onthe item to verify is known. Therefore an automatic scanning system maybe able to locate this letter with a low-resolution scan, then rescanthe letter with a high resolution and finally compensate for therotation.

Matching By Cross-Correlation

2D Cross-Correlation

The cross-correlation is particularly suitable for matching images whichcontain noise. Indeed, it can be used basically for any kind of uniformnoise without special tuning or adaptations.

The formula is shown in Equation 6, where A and B are the two correlatedimages and x and y the position of the result of the sum in thecorrelation picture φ.

$\begin{matrix}{{\varphi_{A,B}( {x,y} )} = {\sum\limits_{x^{\prime} = {- \infty}}^{\infty}\;{\sum\limits_{y^{\prime} = {- \infty}}^{\infty}\;{{A( {x^{\prime},y^{\prime}} )}{B( {{x^{\prime} + x},{y^{\prime} + y}} )}}}}} & {{Equation}\mspace{20mu} 6}\end{matrix}$

This correlation can be computed in the Spatial domain or in the Fourierdomain. In the Fourier domain, the correlation is represented by amultiplication as displayed in Equation 7, where ℑ is the Fouriertransform.φ=ℑ⁻¹(ℑ(A)ℑ(B*))  Equation 7Other Types of Correlations

It should be noted that several variations of cross-correlations existwhich can also perform efficiently in the case of microstructure. Theyare listed below.

-   -   Phase correlation. If only the phase has to be correlated, both        images should be transformed in the Fourier domain. Then the        first image is multiplied with the complex conjugate of the        second image and divided by the modulus of the first image        multiplied with the complex conjugate of the second image.        Finally, the resulting Fourier image is retransformed into the        spatial domain. This is explained by Equation 8

$\begin{matrix}{\varphi = {{??}^{- 1}( \frac{{{??}(A)}*{{??}(B)}^{\prime}}{{{{??}(B)}*{{??}(B)}^{\prime}}} )}} & {{Equation}\mspace{20mu} 8}\end{matrix}$

-   -   Cross-correlation with special padding strategy. Indeed, in the        formula from Equation 6, the indices x′+x or y′+y may exceed the        image size in the finite case and a strategy has to be followed        to define which values should be used in such cases. Some of        these strategies include padding the image with values varying        symmetrically or anti-symmetrically (left/right and up/down)        across their borders, or padding with a fixed value (the mean        value for instance).    -   Cross-correlation by 1D projection. Instead of correlating the        whole 2D picture, it can be projected on one or the other axis,        for example by making the mean of the coefficients of each        column or the mean of the coefficients of each row. Any other        projection method is also possible. The cross-correlation of 1D        signals is much more rapid than the cross correlation of 2D        signals, especially when the image becomes huge. Nevertheless it        is also less robust.    -   Cross-correlation of chosen coefficients. When correlating the        images in the Fourier domain, it is possible to choose the        coefficients which are relevant for the cross-correlation.        Typically the coefficients which have mathematical        characteristics close to white noise can be chosen. Furthermore,        if an image contains an auto-correlating pattern, such as a grid        or a brushing, the frequencies corresponding to the        auto-correlation can be discarded. This induces a speed up as a        fewer number of coefficients are correlated.    -   Filtered cross-correlation in the Fourier domain. If an image        contains an auto-correlation pattern, it is possible to filter        the Fourier transform of this cross-correlation to remove the        repetitive pattern.        Matching Quality

A measure representing the matching quality between 2 normalized imagescan be defined with Equation 9, where M is the maximum of the signal andμ its mean.SNR=20*log(M/μ)  Equation 9

If the images are not normalized, the matching metric should normalizethem as in Equation 10, where m is the minimum of the image

$\begin{matrix}{{SNR} = {20*{\log( \frac{( {M - m} )}{( {\mu - m} )} )}}} & {{Equation}\mspace{20mu} 10}\end{matrix}$

This value characterizes the signal to noise ratio (SNR) of thecross-correlation image and is measured in decibels (dB). When severalcross-correlations are computed (using several images of the same areaor from different areas), the SNR can be very significantly increased bythe multiplication of two cross-correlations aligned on their peaks. Itis also possible to take morphological criteria into account, such asopening and closing operations on the cross-correlation image.

Robustness

In order to avoid false positives, the matching algorithm has to berobust against some modifications of the item. When a big enough dpi ischosen, the image captured can contain a sufficiently big amount ofinformation so that the matching algorithm is robust against slightmodifications. In particular, in the case of watches, the dpi should bebig enough to permit to be robust against adding adhesive or staticlayer, adding dust, damaging by scratching and all other damage that letup to half of the information. It also has to be noticed experimentallythat this method is robust throughout glasses, even watch glass, flat orconvex, with or without anti-reflective coating on it. It is also robustagainst x-y displacement. An offset of 50% still permit to recognize theitem. These considerations are true for the particular case of watchesbut can be applied to any item, which is recognized using theFingerprint technology.

Speed Up of the Technology

Decision Tree

A possibility to speed up the detection process is to perform thematching for image of smaller size to make a first step and thenmatching smaller sets of bigger images. For instance if the image sizeis 1024×1024 and if there are 10,000,000 items in the database,performing all cross-correlations with all templates may take asignificant amount of time (up to 1 hour in some cases). A detectionstrategy consists in performing the detection in several stages.

There are different possibilities to obtain a set of smaller images. Itis possible to use cropped versions of the templates, quantized versionsof the templates or downsampled versions of the templates. Downsamplingis preferred instead of cropping. First, downsampling is more resistantin case of dust or other small variations on the image; second, as thepositioning is very precise, cropping can lead to the image and templateto be completely misaligned. This will not be the case withdownsampling. A first stage is performed with downsampled versions ofthe snapshot and template images and then the next stage use largerversions of the snapshots and templates. This approach is illustrated bydiagram of FIG. 40: cross-correlations are first computed with a set S₀of X₀ templates using an image size of 2^(n)×2^(n) pixels (the samemethod may of course be used for non square images or non integer powerof 2 image sizes). A number X₁₂ of cross-correlation images have an SNRover a given threshold t₁ and are then selected as candidates for asecond test with larger image of size 2^(n+1)×2^(n+1). The sameprocedure continues with threshold t₂ and with increasing image sizesand thresholds until the original image of size 2^(n+x)+2^(n+n) isreached resulting in one unique candidate X_(x2)=1 which corresponds tothe snapshot. Such strategy is not limited to the case ofcross-correlation and can potentially be applied with any matchingmetric.

A practical example is given in order to illustrate this process. In anexperiment n=3 and x=10 were used for cross-correlations ofX₀=10,000,000 templates with a test image. The following number ofcandidates was then obtained: X₁₂=112539, X₂₂=1234, X₃₂=2, X₄₂=1, X₅₂=1.

Depending on noise characteristics, downsampling down to 8×8 images sizecan easily be reached.

If the correlation is done in the Fourier domain, the coefficients canbe stored in a database in an efficient way. It is generally admittedthat downsampling an image in the spatial domain will result in a cropin the Fourier domain. Therefore only the coefficients of set S_(x) arestored in the database. Then for the matching of sets S₀ to S_(x−1),only some of the coefficients are retrieved from the database. To beaccessed efficiently they are split between the different columns. Thecoefficients for the 2^(n)×2^(n) images can be stored in one column.Then, instead of storing all the coefficients of the 2^(n+1)×2^(n+1)images, only the remaining ones up to this size can be stored in thenext column. The coefficients that are stored in each column 491 of thedatabase table 493 are represented by the black area on FIG. 49. Thefigure shows that column 1 has only one coefficient (the average valueof the image), the column 2 has the 3 following coefficients, the column3 has 12 coefficients, etc. . . . . This approach enables to optimizethe required bandwidth for transferring data (492) from the database onthe hard disk to the CPU. In fact, all the coefficients of set S₀ aretransferred but then only the remaining coefficients from the relevantrows are transferred. A new line 494 is allocated in the database foreach template image. Furthermore the multiplied coefficients of therelevant correlations can be stored in order to avoid redundantmultiplications. In fact only the coefficients that are displayed inblack should be correlated.

Bayesian Network

A speed up can also be obtained by using a theory based on Bayesprobabilities. The notations are the same as those of FIG. 40. Let P(G)be the probability that an item is genuine. For a set S_(i) ofcross-correlation, if the SNR is above the given threshold t_(i+1), thenthe probability for the image to be already recorded is denoted a. Thisis modeled by Equation 11.P(G|SNR _(i) >t _(i+1))=ai=0, . . . , x−1  Equation 11

It can be stated that if the SNR is some fraction lambda betweent_(i+1), and t_(i+2), then the probability for the image to be alreadyrecorded is b and b>a. This is modeled by Equation 12.P(G|SNR _(i) >t _(i+1)+λ(t _(i+2) −t _(i+1)))=bb>a,i=0, . . . , x−1λε[0,1]  Equation 12

All the following assumptions are formulated:

-   -   The higher the SNR of a cross-correlation of images of a given        size, the higher the SNR of the cross-correlation of images of        bigger sizes and the higher the probability of the image to be a        recorded one. This is explained by Equation 13.        P(G|SNR _(i) >t _(i+1))=a        P(G|SNR _(i+j) t _(i+j))=b        p(G)=c        i=0, . . . , x−1        j>0|i+j≦x        0≦a≦b≦c≦1  Equation 13    -   For a given set of cross-correlation, if the SNR is under a        given threshold, then the probability for the image to be        already recorded is 0. This is modeled by Equation 14        P(G|SNR _(i) <t _(i+1))=0        i=0, . . . , x−1  Equation 14    -   For the cross-correlation from S_(x), if the SNR is above the        predefined threshold, then the probability for the image to be        already recorded is 1.        P(G|SNR _(x) >t _(x+1))=1  Equation 15

The speed up can be obtained the following way. First all the items ofset S₀ are correlated together. For each item, if the probability to begenuine is below a, the item is discarded. If it is between a and b, itis put in a set of possible match to be correlated in S₁ as for thedecision tree algorithm. If the probability to be genuine is more thanb, then the picture is directly correlated at higher sizes up to size2^(n+x)+2^(n+x). If it is the good match, the algorithm stops. Else itcontinues to correlate templates of set S₀, until all have beencorrelated. Then if the match is still not found the same algorithm isapplied for the following sets S₁ up to S_(x).

Best Rank

This method is a hybrid one between Decision tree and Bayes networks.The notations are those of FIG. 40. Experimental results show that, fora given set of templates, the SNR obtained with low resolution images(typically those of set S₀) may significantly differ between imageditems. Furthermore, the rank of the good match is not inevitably thefirst. Nevertheless, the rank has a smaller variation than the SNR.Experimentally it has been tested to be always in the 5% first. So itcan be assumed that if the rank for a given size of one template isgood, there is a higher chance of a match.

So sets can be created by taking into account the templates with highestranks. FIG. 50 is useful to understand this theory. The followingnotations are used:

-   -   x is the number of sets, as shown in FIG. 40,    -   p is the current set used for cross-correlation.    -   i is the current iteration    -   C′_(ixp) is the number of templates to take at iteration i from        set p, for the next set p+1.

The C′_(ixp) best templates are taken at each step. In fact as some ofthe best templates have already been correlated during the precedingiteration, there is no need to correlate them again. C_(ixp) is biggerfor smaller size images than for the bigger ones. If after oneiteration, the good match is not found, all the C_(ixp) are increaseduntil the good match is found or until a decision is taken that theimage is not in the database. As the size of the image has a geometricalgrowth, the set of remaining templates at each set should also follow ageometric law. The idea is to have an increasing common ratio for thegeometric progression. Two things are important with this method: thestop criterion as well as the increasing law of the common ratio of thegeometrical progression. A geometrical law can be chosen to increase thecommon ratio of the geometrical progression. The stop criterion ischosen so that the application stops before correlating all thetemplates with a size of 2^(n+1)×2^(n+1). In fact it is assumed that, ifall the templates of size 2^(n+1)×2^(n+1) are correlated, there was noneed to use the templates of size 2^(n)×2^(n). More precisely theC_(ixp) are computed as in Equation 16 until i<j. The first linecomputes the number of templates to take at each step. It corresponds tothe number of templates as computed in the second line minus thetemplates that have already been taken in the preceding iterations. Thesecond line computes the geometrical progression with a common ratio ofa. The power corresponds to the iteration number (i) as well as thenumber of set (x) and the current size (p). The third line simplyformulates that at the first iteration no templates have already beencorrelated, therefore the number computed by the second line should betaken into account. The fourth line represents the stop criterion. Ittells that the algorithm should stop if S₁≧S₀.C′ _(ixp) =C _(ixp) −C _((i-1)xp)C _(ixp) =a ^(i(x-p))C′_(0xp) =C _(0xp)i=0, . . . , j, j|C _(jx1)≦Card(S ₀)  Equation 16

For example if a=2 and x=5, the following number of templates C_(ixp)should be taken at each step. Each row is representing an iteration i.The columns represent index of the set of images. It should be remarkedthat the last column always contains only one template, as only onematch can be found. In the first row, at i=0, only the best template iscorrelated. In the next row, at i=1, 32 templates from S₀ are taken tocorrelate in set S₁. It can be remarked that the number of templatetaken from S₀ is growing rapidly. The coverage of the database can beseen in FIG. 50.

TABLE 1 p i 0 1 2 3 4 5 0 1 1 1 1 1 1 1 32 16 8 4 2 1 2 1,024 256 64 164 1 3 32,768 4,096 512 64 8 1 4 1,048,576 65,536 4,096 256 16 1Neighbors Classifiers

This theory is based on the transitivity of the correlation. It is truethat if an image A correlates completely with an image B and if theimage B correlates completely with an image C, then A correlatescompletely with C. But, if an image A doesn't correlate with an image Band if the image B doesn't correlate with an image C, then nothing canbe told about the correlation of A and C. The question is then if Acorrelates to some degree with B and B correlates to some degree with C,what can be told about the correlation of A and C? It can be assumedthat the highest the degree of correlation of A and B and of B and C,the highest the probability that A and C also correlate. Therefore, thegoal is to compute subset of templates that are well correlatingtogether. Then, for the images of group S₀ from FIG. 40 instead ofcorrelating the snapshot with all the templates, it is correlated onlywith the representative of its group. Then a certain number of groupsare chosen and the best rank method is used for the other set of images;for S₁ up to S_(x) of FIG. 40.

Robustness

Noise

Noise Evaluation

Some additional noise can induce a common pattern between all thecaptured images. Therefore the correlation of the noise can induce afake positive. The noise of the camera is composed of different noises:the thermal noise and the CCD sensor noise. The thermal noise is not aproblem in the present case because it is random and changes at eachimage capture. The sensor's noise is induced by the difference of lightresponse of each pixel of the CCD sensor. Moreover, dust can be presentdirectly on the CCD sensor or on one of the mirrors or lens of theoptical way of light. This noise is a constant and that's why it shouldbe removed. It is quite difficult to characterize. Nevertheless, it isquite easy to approach by doing the mean of some captured images.

Noise Removal

Different methods can be used to remove the noise. They are listedbelow.

-   -   If it is on an identified location, it can be padded either with        the mean value of the image or with a random noise at the mean        value of the image.    -   If the noise is spread out everywhere on the image and cannot be        localized, removing the correlation induced by the noise instead        of the noise itself can be a very good compromise. FIG. 44 shows        one possibility of removing the correlation due to the noise of        the camera. 441 shows the normalized noise correlation and 442        is the normalized correlation between the snapshot and its        corresponding template. It can be seen that there are two peaks        of correlation; the sharp centered one (445) is due to the noise        and has a value of λ. The other one is the real peak. The mean        of the correlation is μ (446). 443 represent the noise        correlation, which is stretched in order to correspond to the        noise in the correlation between the snapshot and the template.        It has the same mean μ (448) and the same value for the peak due        to the noise λ (447). 444 is the normalized subtraction from the        correlation between the snapshot and the template. The        stretching is obtained by the formula of Equation 17, assuming        that μ is the mean of 442 and λ is the value of the noise        correlation in 442.

$\begin{matrix}{{sv} = {{\frac{v - m}{M - m}( {\lambda - \mu} )} + \lambda}} & {{Equation}\mspace{20mu} 17}\end{matrix}$Spectrum Whitening

Sometimes, it can be difficult to find a homogenous zone and thecorrelation can reveal macroscopic details instead of microscopic ones.So the Spectrum of the Fourier transform can be whitened in order toremove macroscopic details. There are several approaches to whitensignals, which are listed below.

-   -   Official whitening. The usual way of whitening a signal is        described by Equation 18. In this equation, x is the signal, μ        is the mean of x, Δ is the diagonal matrix of eigenvalues of the        covariance matrix of x, E is the orthogonal matrix of        eigenvectors of the covariance matrix of x and w is the        resulting whitened signal        w=Δ ^(−1/2) E ^(T)(x−μ)  Equation 18    -   Removing some Fourier coefficients. In order to whiten the        spectrum, some coefficients of the Fourier transform of the        signal can be discarded.    -   Any other method to whiten the spectrum can be used.        Rotation and Scale Compensation

Some imaging devices, such as a mobile phone, do not enable toreposition the item in the same orientation than the template.Furthermore a scale factor can distinguish the image from the template.It is then possible to store multiple templates with different scalefactors and different rotation angles. For downsampled images, such astemplates from set S₀, if the images have different angles or scalefactor, the Fourier transform is quite the same. On the contrary for setS_(x), if the images have different angles or scale factors, thedifference between images is big. Therefore the idea is to have anincreasing number of templates at each size of images as in FIG. 52. Itshows the tree representing the different scale factors or rotationfactors for an item. 521 shows that at low resolution, the templateswith different scale factors have the same Fourier transform. Then, whenthe resolution increases, the different scale factors become distinct.At high resolution (522), there is one Fourier transform per scalefactor. This approach is also true for changes in rotation factor. If ahierarchical approach is used to retrieve the good match, this willimprove the robustness with only a marginal speed decrease becausecardinal of set S₀ (whose correlations with acquired image accounts formost of the computation time) is simply unchanged.

Finally, it should be noted that the applications of above describedtree approach is not limited to rotation or scale compensation. Thisapproach can be used to any compensate or progressively refine manyimage parameters which are critical to the cross-correlation process.For instance, such approach can also be used at translation level. Inthis case, a first correlation is performed with a cropped version ofthe imaged object and the size of the crop size is progressivelyincreased in order to refine the SNR. The tree refinement strategy shownin FIG. 52 can also be used in this case if following modifications aredone:

-   -   The cropping operations are performed by sub-sampling the        Fourier images of each tree node (for instance by keeping only        even or odd coefficients).    -   Crop location at a given level in the tree is given by the        position of the cross-correlation peak at the previous level        (i.e. at the lower resolution).        Other Techniques Increasing Robustness

The robustness can be increased by several other methods listed below.

-   -   In the case of manually handled imaging device, the device may        display a template of the product to be imaged in order to aid        the operator to position precisely the imaging (thus insuring        rotation-translation-scale-perspective-etc invariant imaging)    -   Rotation may be compensated by using cross-correlation in the        Fourier domain 532 of a ID signal. Such signal is obtained by        sampling the Fourier modulus over a circle 531, on both the        template and acquired image as shown in FIG. 53. The        cross-correlation position indicates the angle of rotation.    -   The SNR of the good match can be increased by correlating the        mean of a set of snapshots of the same item with the template.    -   It can also be increased by correlating a set of snapshots of        the same item with the templates and then align them on their        correlation peak and then averaging them.    -   Any other method enabling to highlight the microscopic        structures is considered as a robustness improvement of the        technology

1. Method to identify an object comprising a parameter settings phase,an acquisition phase and an identification phase, the parameter settingphase comprising the steps of: defining for a given set of objects, aresolution, a type of non-coherent light illumination and a location,called region of interest, for the acquired image for which the object'smicrostructure image contains noise, the acquisition phase comprisingthe following steps, for each object to be later identified: digitallyacquiring a two-dimensional image of the object according to parametersettings through sampling on a uniformly spaced orthogonal grid of atleast one color component, applying a flattening function on saidtemplate in order to produce a flattened template by removingmacroscopic color variations, generating at least one downsampledtemplate version of the flattened template, storing in a referencedatabase the downsampled template version and the flattened template,the identification phase comprising the following steps, for the objectto be identified: digitally acquiring a two-dimensional snapshot imageaccording to the same parameters as the template image, applying to thesnapshot image the same flattening function as the one applied to thetemplate image in order to produce a flattened snapshot image,generating at least one downsampled version of the flattened snapshotimage, cross-correlating the downsampled version of the flattenedsnapshot image with the corresponding downsampled templates versionstored in the reference database, and selecting the templates accordingto the value of the signal to noise ratio of said cross-correlation, forthe selected templates, cross-correlating the flattened snapshot imagewith the flattened template stored in the reference database, and thusidentifying the object by finding the best corresponding template whichsignal to noise ratio value of said cross-correlation is above apredefined threshold.
 2. Method of claim 1, wherein the flatteningfunction is calculated using the difference between the image and thelow-pass version of said image.
 3. Method of claim 1, wherein theacquisition or the identification phase comprises a prior step ofapplying rotation compensation by acquiring another image of the objectand applying a Fourier transform to the said image in order to measureanisotropic information and use it to compute the rotation compensationangle.
 4. Method of claim 1, wherein the signal to noise ratio iscalculated using the cross-correlation of the template and the snap-shotimage, the signal to noise ratio being the ratio between the peak valueof the cross-correlation and the mean value of said cross-correlation.5. Method of claim 1, wherein it comprises the step of converting thetwo-dimensional template in one-dimensional template by summing theintensity along each line, and converting the snapshot image intoone-dimensional in one-dimensional template by summing the intensityalong each line image by summing the intensity along each line. 6.Method of claim 1, wherein the parameter settings phase and theidentification phase are modified as follows: the parameter settingsphase comprise an additional step consisting in defining for a given setof objects at least one visual feature whose distance to the X/Yposition of the region of interest is known, creating a mask image byacquiring a two-dimensional image of feature points, the identificationphase comprising preliminary steps of: digitally acquiring atwo-dimensional image of the feature points of the object,cross-correlating the acquired image with mask image in order toretrieve the location of the region of interest where the snapshot imagewill be acquired.
 7. Method of claim 1, wherein the template identifiedcan serve to authenticate a plurality of objects having the samemicrostructure.
 8. Method of claim 1, wherein no template identifiedallows detecting counterfeited object.
 9. Method of claim 1, wherein theacquisition and identification phases are modified, the acquisitionphase generating at least two downsampled versions of the templatehaving different resolutions, the identification phase including thefollowing steps: generating the same number of downsampled versions ofthe snapshot having different resolutions iteratively selectingtemplates by a cross-correlation between a downsampled version of thesnapshot image and the template of the same resolution according to thevalue of the signal to noise ratio of said cross-correlation, startingfrom the lowest resolution and ending at higher resolutions.
 10. Methodof claim 1, wherein the parameter settings, acquisition andidentification phases are modified as follow in order to successfullymatch corresponding template and snapshot images having different anglesof rotation: the parameter settings phase having an additional step ofdefining the shape and location of a circular donut inside the region ofinterest, the acquisition phase comprising the steps of: extracting fromthe acquired image the circular donut region, unwarping the donut in arectangular shape in order to define the template image, theidentification phase comprising the steps of: extracting from theacquired image the circular donut region, unwarping the donut in arectangular shape in order to define the snapshot image.
 11. A device toidentify an object comprising: image acquisition means; storage means,said storage means comprising a plurality of template entries, at apredefined resolution, type of non-coherent light illumination andlocation, for an acquired image for which the object's microstructureimage contains noise, each template comprising at least two versions ofthe template, a full resolution flattened template and a downsampledversion of the flattened template, said device comprising means toproduce a flattened snapshot of the acquired image, means to produce atleast one downsampled version of the flattened snapshot, means tocompare the downsampled version of the flattened snapshot with thedownsampled version of the flattened template using cross-correlationmeans to produce a first signal to noise ratio; and selection means toselect the downsampled flattened template according to the value of thefirst signal to noise ratio; wherein the means to compare furtherperforms cross-correlations of the full resolution flattened snapshotwith the full resolution flattened templates corresponding to thepreviously selected downsampled version templates to produce a secondsignal to noise ratios and to use the second signal to noise ratios toidentify one template as corresponding to the acquired image.